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PATENT 

ATTORNEY DOCKET NO. 213187-00002 

IN THE UNITED STATES PATENT AND TRADEMARK OFFICE 
BEFORE THE BOARD OF PATENT APPEALS AND INTERFERENCES 

Applicants): Bates et al. ) 

Application No.: 09/679,391 ) 

Filed; October 3, 2000 ) 

far. Method and Apparatus for Associating the Color of an ) 
Object with an Event ) 

Group An Unit: 2676 > 

Examiner: Blackman, Anthony i > 

Customer No.: 27160 

Confirmation No.: 6681 

APPLICANTS' BRIEF ON APPEAL 

Mail Stop Appeal Brief-Patents 
Commissioner for Patents 
P.O.Box 1450 
Alexandria, VA 22313-1450 

REAL PARTY IN INTEREST 
The real party in interest js Creatier Interactive, LLC by virtue of an Assignment from the 
U.S. Bankruptcy Court to Creatier Interactive LLC, recorded on Reel/Frame 015036/0672. 

RELATED APPEALS AND INTERFERENCES 

There are no other appeals °r interferences known to the Appellants or the Appellants' 
representative, which are believed to directly affect or be directly affected by or have a bearing 
on the Board's decision in the pending appeal. 
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STATUS OF CLAIMS 
Claims 31-38 are pending. The final rejection of claims 31-38 form the basis for this 
appeal Claims 31-36 have been rejected under 35 US.C, §102(e) as being anticipated by 
Rangan et al„ U.S. patent no, 6,198,833. Claims 37 and 38 have been rejected un 35 U.S.C. 
§l03(a) as being unpatentable over Rangan et al. ? U.S. patent no. 6,198,833 in view of issadore- 
Barreca et al, U.S. Patent No. 6,205,231. 

STATUS OF AMENDMENTS 

All amendments have been entered. The claims as currently amended are attached as an 
Appendix. 

SUMMARY OF CLAIMED SUBJECT MATTER 
The present invention relates to an automatic authoring system for interactive television. 
In such interactive television systems, it is desirable to *tag" items of interest that may appear in 
the video frames which may be of interest to a viewing audience. Such interactive television 
systems allow the user to click on the tagged item during the broadcast for additional information 
on the item of interest In order to provide such capability, items of interest have to be tagged in 
all video frames where the items of interest may appear- In a NTSC broadcast, video frames are 
broadcasted at a rate of about 30 frames per second. Thus, a one-hour broadcast would involve 
108,000 video frames. Tagging the 108,000 video frames for the various items of interest is an 
arduous task, to say the least- In order to solve this problem, automatic authoring systems have 
been developed. These systems allow a user to tag items of interest in a single video frame and 
the system automatically locates and tags these items in succeeding video frames, 

The present invention relates to a relatively simple method and apparatus for tagging 
items of interest in video frames by associating a color 120 of an object with an event. In 
particular, the system determines the locations within each video frame where a selected item 
may appear. The system determines the color value of the selected item by its digital color 
mathematic properties. For example, one embodiment of the invention utilizes the RGB 
standard which separates the red 220, green 230, and blue 240 properties as the color as 
numerical values 250. Another embodiment uses the HSV characteristics of the color which 
define the hue 320, saturation 330 and value 340 and components of color. 

4 
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Once an item of interest is determined in an initial frame, the system then automatically 
locates ibe selected item based on the color mathematical characteristics in the remaining frames 
in the broadcast. These items are then tagged and linked to an event, such as a URL. One 
embodiment of the invention uses an edge detection scheme to define the boundaries for the 
colors of the selected object. An important aspect of the present invention is that the invention 
utilizes a relatively simple process for otherwise completing a rather arduous task, 

GROUNDS OP REJECTION TO BE REVIEWED ON APPJEaL 

I. The rejection of claims 3 1-36 as being anticipated by Rangan et ai, U.S. patent 
no. 6,198,833 (hereinafter referred to as "Rangan.") 

II. The rejection of claims 37 and 38 as being unpatentable over Rangan et al., U.S. 
patent no. 6,198,833 in view of Issadore-Barreca et al., U.S. Patent No. 6,205,231 (hereinafter 
referred to as "Issadore-Barreca,") 

ARGUMENT 

I- REJECTION OF CLAIMS 31-36 UNDEft 35 U.S.C. §102(E) AS BEING 
UNPATENTABLE OVER RANGAN fiT AL 

It is respectfully submitted that the claims 31 «36 stand or faU together. Claims 31 and 32 

have been rejected under 35 U.S.C. § 102(e) as being anticipated by Rangan, et al, U.S. Patent 
No- 6,198,833- In order for there to be anticipation, each and every one of the elements of the 
claims must be found in a single reference. It is respectfully submitted that Claims 3 1 and 32, as 
well as Claims 33-36, include elements clearly not suggested or disclosed in the Rangan, et al. 
patent. For example, Claim 31 recites, in combination, a method which includes a step for 
determining the location where an action by a pointing device has occurred. This claim also 
recites the step of determining a color value with the selected location and automatically 
associating an event with the color value in succeeding video frames. Claim 32 is similar. 
Claim 35 recites a system in which the locations of an object selected by a pointing device are 

5 
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determined. This claim also recites determining a color range for the selected object and 
associating an event with the color range in succeeding video frames. Claim 34 is similar. 
Claims 35 and 36 are similar but relate to a color pattern- Finally, Claims 37 and 38 are similar 
but relate to an edge of a selected object. The Rangan patent does not disclose such elements. 

Rangan, on the other hand, tracks a rectangular wire frame, having a fixed size around the 
location where the pointing device was exercised by comparing the color of individual pixels in 
the fixed wire frame from one frame to the next- Rangan does not define an object based on its 
color, nor does it dynamically track the object based on color as in the claims at issue. Rangan 
trades objects based on the movement of individual pixels from frame to frame within a wire 
frame, This movement tracking is based on the initial wire frame location which is tracked in 
each frame and does not automatically associate a color with an oty'ect as in the claims at issue. 
Tracking movement of the wire frame in Rangan demands testing of individual coordinates 
against their counterpart from the previous frame. This testing requires significant data to be 
computed at each frame. Further, it is respectfully submitted that Raugau's color signature are 
not representative of a continuous color pattern. In particular, Rangan's color signature records 
the color value of every pixel in a fixed area rather than detecting a color pattern based on a 
region of interest. As such, it should be clear that claims 31-36 disclose elements clearly not 
disclosed or suggested by Rangan. Thus, the Board is respectfully requested to reverse this 
rejection. 
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U. REJECTION OF CLAIMS 37-38 UNDER 35 U.S.C. qi 03f A) AS BEING 
UNPATENTABLE OVER RANGaN ET AL. IN VIEW QF 
ISSADQRE-BARRFCA et al. 

Claims 37 and 38 recite, in combination, a method and system for automatically 
identifying a bounded object based on color characteristics in a series of video frames and 
associating an event with said object. It is respectfully submitted that claims 37 and 38 are not 
suggested by Rangan or Issadore-Barreca. As discussed above, Rangan does not track objects in 
the same maimer as the system in the current claims. In particular, Rangan tracks the movement 
of an object of fixed shape by comparing the color of individual pixels in a fixed matrix from one 
frame to the next. Nowhere does Rangan disclose or suggest tracking an object based solely on 
its color, nor does it dynamically change the shape and size of the object based on color. 

The Issadore-Bamrca, et al. patent was cited for disclosing an edge detection scheme. 
However, the Examiner has failed to show thai an edge detection scheme would be compatible 
with the fired wire frame tracking method taught by Rangan- Indeed, both systems would appear 
to be incompatible with each other. 

Also, it is respectfully submitted that the Examiner has failed to establish a prima facie 
case of obviousness as set forth in MPEP § 2142 and § 2143. Jn order to establish a prima facie 
case of obviousness, three criteria must be met as set forth in MPEP § 2143. 

"First, there must be some suggestion or motivation, either in the references 
themselves or in the knowledge generally available to one of ordinary skill in the 
an, to modify the reference or to combine reference teachings. Second, there 
must be a reasonable expectation of success. Finally, the prior art reference (or 
references when combined) must teach or suggest all the claim limitations. 

The teaching or suggestion to make the claimed combination rea$onable 
expectation of success must both be found in the prior art, not in the Applicant's 
disclosure." 

It is respectfully submitted that the examiner is combining two references in support of the 
rejections without a showing that the motivation for the combination was suggested by the 
reference itself. Moreover, none of the references, either singly or in combination, teach all of 
the elements of the claims. For example, all of the claims recite automatically associating an 

7 
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object with its color in a plurality of video frames. None of ibe references cited disclose such an 
automatic authoring system- Moreover, there is no reasonable expectation that the combined 
references would succeed in an interactive TV application. For all of the above reasons, the 
Board is respectfully requested to reverse the Examiner's rejection of claims 37 and 38. 
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CONCLUSION 



The Board is respectfully requested to reverse the rejection of the claims by the 
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APPENDIX A 
CLAIMS ON APPEAL 

31. A method for automatically identifying an object in a plurality of video frames 
and associating the object with an event comprising: 

determining a location in one of said video frames where m action by a pointing device 
has occurred defining a selected location; 

determining a color value of said selected location; and 

automatically associating an event with said color value of said selected location in said 
one video frame and automatically associating events with said color value in succeeding video 
frames. 

32. A system for automatically identifying an object in a series of video frames and 
associating an event with said object, the system comprising: 

a system for determining a location in one video frame where an action by a pointing 
device has occurred defining a selected location; 

a system for determrniag a color value associated with said selected location in said one 
video frame; and 

a system for automatically associating an event with said color value in succeeding video 

frames. 

33. A method for automatically identifying an object in a plurality of video frames 
and associating the object with an event comprising: 

determining the coordinates of an object in a video frame that is selected by a pointing 
device defining a selected object; 

10 
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determining a cojor range for said selected object; and 

automatically associating an event with said color range of said selected location in said 
one video frame and automatically associating events with said color value in succeeding video 
frames. 

34. A system fox automatically identifying an objec? in 3 series of video frames and 
associating an event with said object, the system comprising; 

a system for determining the coordinates of an object in one video frame that has been 
selected by a pointing device defining a selected object; 

a system for determining a color range for said selected object in said one video frame; 

and 

a system for automatically associating an event with said color range in said one video 
frame and succeeding video frames. 

35. A method for automatically identifying an object in a plurality of video frames 
and associating die object with an event comprising: 

deterxnining the coordinates of an otyeci selected in one of said video frames by a 
pointing device defining a selected object; 

determining a color pattern for said selected object; and 

automatically associating an event with said color pattern of said selected location in said 
one video frame and automatically associating events with said color pattern in succeeding video 
frames. 

36. A system for automatically identifying an object in a series of video frames and 
associating an event with said object, the system comprising: 

U 
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a system for deteroiining the coordinates of an object in one video frame selected a 
pointing device defining a selected object; 

a system for determining a color pattern for said selected object in said one video frame; 

and 

a system for automatically associating an event witb said color pattern in said one video 
frame and succeeding video frames. 

37, A method for automatically identifying an object in a plurality of video frames 
and associating the object witb an event comprising: 

determining the coordinates along one edge of m object in one of said video frames 

selected by a pointing device defining a selected object; 

determining the color values along said edge of said selected object; and 

automatically associating an event with said color values aJong said edge in said one 

video frame and automatically associating events with said color values in succeeding video 

frames associated with the color values. 

38. a system for automatically identifying an object in a series of video frames and 
associating an event with said object, the system comprising: 

a system for determining the coordinates along one edge of an object in one video frame 
selected by a pointing device defining a selected object; 

a system for determining the color values along said edge of said selected object in said 
one video frame; and 

a system for automatically associating an event with said color values in said one video 
frame and succeeding video frames. 
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APPENDIX B 
EVIDENCE 

None. 
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APPENDIX C 
RELATED PROCEEDINGS 



None. 
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